Separating illuminant and surface reflectance spectra from filtered trichromatic camera measurements

We show how illuminant and reflectance spectra can be accurately measured or separated, up to a multiplicative factor, at each pixel of a scene by using a CCD digital camera instead of a spectroradiometer. In order to obtain suitable spectra of both illuminants and reflectances in a scene, we may use a 6channel measure from the digital 3-channel RGB camera. This is accomplished by taking two images of the scene, and using a colour filter during the second. No other practical or theoretical restrictions are needed to apply this separation algorithm, which is based on the validity of low-dimensional linear models for representing illuminant and reflectance spectra. Introduction A colour signal [1], or radiance spectrum, can be defined as any function which represents the spectral power distribution (SPD) of the product of the spectral reflectance of one pixel of an object and the SPD of the light source that illuminates it. The ability to separate the surface reflectance spectrum from the illuminant spectrum at each pixel is useful for many tasks, and it is still one of the unsolved problems in multispectral colour science. For example, surface spectral reflectance data can be used to classify minerals [2] or to simulate the colour appearance of an object under illuminant changes, which would be desirable for visually guided robots, automatic terrain classiffication by remote sensing or for better colour reproduction in colour displays, among many other applications [3]. Achieving this spectral signal separation by means of digital cameras and multispectral techniques would be specially useful, leading the way to use portable digital cameras -instead of spectrometersto render high spatial resolution colour images. In this work we use the Wiener estimation method [4] to obtain the spectral colour signal of a scene from the simulated responses of a trichromatic camera coupled with a filter, instead of using a spectrometer [2,3]. We then separate this radiance spectra into spectral reflectance and illuminant components by using the method proposed by Ho et. al. [3], which has been also used by other authors but always making use of spectral radiance measurements from a spectrometer [2,3] instead of using a trichromatic camera and a filter. We find that spectral reflectance and illuminant can be accurately obtained at each pixel, up to a multiplicative factor, from trichromatic camera measurements of a scene by making use of finite-dimension linear models for reflectance and illuminant spectra. Method The RGB digital camera had spatial resolution 1280 × 1024 pixels (QImaging, model Retiga 1300, QImaging Corp., Canada) and 12 bits intensity resolution per channel. Several hyperspectral colour signal data from various scene fragments [5] were used as training spectra for the Wiener estimation method to obtain the matrix relating colour signals and camera responses (recovery matrix). The “matrix-training set” S, used to obtain the recovery matrix, was formed from 30 different fragments taken from 30 scenes, each fragment of size 151 × 151 pixels. The six camera responses ri to the color signals (i = 1,..,6 for red, green, and blue sensors, respectively and three additional responses for the camera coupled with a blue plastic filter) were computed. The six camera responses for each pixel formed the response matrix R for the entire colour signal set. The recovery matrix D was then computed from the pseudoinverse of R (denoted by superscript +) by